Harvester with automated capabilities

ABSTRACT

Systems and methods here may include a vehicle with automated robotic subcomponents for harvesting delicate agricultural items such as berries. In some examples, the vehicle includes a targeting subcomponent and a harvesting subcomponent. Which may utilize multiple cameras to create three dimensional maps of foliage and targets. In some examples, the targeting subcomponent includes automated or semi-automated harvesting targets to be mapped and passed to the harvesting subcomponent. In some examples, the harvesting subcomponent includes vacuum features and padded spoons to detach the target agriculture from the stem.

CROSS REFERENCE

This application relates to and claims priority to U.S. Provisionalapplication 62/796,319 filed Jan. 24, 2019 the entirety of which ishereby incorporated by reference.

TECHNICAL FIELD

This application relates to the field of automated agriculturalharvesting equipment using robotic assemblies, mobile harvesting units,remote harvesting, target tracking, and various combinations of relatedtechnologies.

BACKGROUND

The agriculture industry is highly reliant on human pickers to harvest anumber of produce, including berries such as strawberries. The reasonhuman pickers are still used today, despite the technologicaladvancements available, is because of the difficulty of identifying atarget such as a berry in a field, that is ready to be picked, reachingthrough the foliage of the plant to grasp that berry, and then carefullyremoving that berry without damaging it, to package and sellimmediately.

Current automatic harvesting of such delicate and difficult to graspagricultural targets such as berries, while operating in a harsh outdoorenvironment did not exist before this application.

SUMMARY

Systems and methods here may include a vehicle having varioussubcomponents for harvesting delicate agricultural items such asberries. In some examples, the subcomponents may be automated. In someexamples, the vehicle may include a targeting subcomponent and aharvesting subcomponent. In some examples, the targeting subcomponentutilizes multiple cameras to create three dimensional maps of the targetand target areas sometimes including the agricultural foliage. In someexamples, the targeting subcomponent may include any of various cameras,sensors, or other targeting features to locate and map targets in anautomated or semi-automated manner. The system may then determinecoordinates of the mapped targets to be passed to the harvestingsubcomponent. In some examples, the harvesting subcomponent may includevacuum features which help a nozzle attach to an agriculture target forharvesting. In some examples, the harvesting subcomponent includespadded spoons to aid in removal of the targeted agriculture from theplant, including in some examples, a stem.

Systems and methods here may include a harvesting vehicle system with avehicle with a targeting subcomponent and a harvesting subcomponent, thevehicle including at least one motor in communication with wheelsmounted to traverse planter bed rows, and alternatively or additionally,the vehicle includes a computing device with a processor and a memory,the computing device in communication with multiple sensors configuredto generate and send sensor data regarding agricultural targets to thecomputing device, wherein the computing device configured to map theagricultural targets using the sensor data, the harvesting subcomponentin communication with the computing device, the harvesting subcomponentincluding a robotic arm with picker head assembly, the picker headassembly including a vacuum hose in communication with a compressor, thehose terminating in a bellows end and spoons configured to pinchtogether to remove targets using data received from the computerregarding mapped targets. In some example embodiments, alternatively oradditionally, a conveyor belt system is included, mounted to theharvesting subcomponent, the conveyor belt system configured to receiveand move targets from the harvesting subcomponent to a packing area ofthe system.

Alternatively or additionally, some examples include a turbine andvacuum assembly mounted to a robotic arm of the harvester subcomponent,the turbine and vacuum assembly in communication with the computer,configured to chop and suction material using data from the computer andmap. In some examples, alternatively or additionally, the harvestingsubcomponent includes a picker head assembly with multiple picker headsmounted to one robotic arm, the picker heads configured to rotate toharvest and deposit agricultural targets. In some examples,alternatively or additionally, a foliage management subcomponent ismounted to the harvester subcomponent, the foliage managementsubcomponent including at least two belt drives in communication withthe computing device, the belt drives including a belt configured torotate on the belt drives, the belt configured to interact with foliageof the agricultural targets, and bend the foliage to reveal agriculturaltargets for the sensors. In some examples, alternatively oradditionally, the belt drives are synchronized with the at least onemotor by the computing device, to move the harvester wheels, such that arelative speed of the foliage and a portion of the belt that isconfigured to contact the foliage, is close to zero.

In some examples, alternatively or additionally, the foliage managementsubcomponent includes a suction blower mounted above the foliagemanagement subcomponent, the suction blower configured to suck ambientair from around the target foliage, up and away from the foliage. Insome examples, alternatively or additionally, wherein the foliagemanagement subcomponent includes a second set of at least two beltdrives and a belt mounted around the at least two belt drives,configured to rotate on the belt drives, wherein the two combined beltsconfigured to squeeze the foliage to reveal agricultural targets. Insome examples, alternatively or additionally, the second set of beltdrives are synchronized with the at least one motor by the computingdevice, to move the harvester wheels, such that a relative speed of thefoliage and a portion of the belt that is configured to contact thefoliage, is close to zero. In some examples, alternatively oradditionally, at least one air blower is mounted on the foliagemanagement subcomponent, the air blower configured to blow air towardthe foliage to clear debris. In some examples, alternatively oradditionally, the robotic arm includes a second vacuum assemblyincluding a hose, configured to receive the agricultural targets fromthe picker head assembly and remove the targets from the harvestersubcomponent to a packing area of the system. In some examples,alternatively or additionally, a back end computing system is incommunication with the harvester computer, the back end computing systemconfigured to allow human users to review sensor data and designateagricultural targets for the harvester subcomponent to harvest. In someexamples, alternatively or additionally, the computer uses neuro networklogic to make preliminary determinations of targets using the sensordata. In some examples, alternatively or additionally, the computer mayallow the human user to determine an angle of attack for the picker headto harvest an agricultural target, using the sensor data.

Systems and methods here include harvesting agriculture, including atraversing machine with at least two robotic articulating arms attachedto it, at least two wheels attached to it, and a computing system withat least a processor and memory, the traversing machine including atleast two wheels or tracks configured below an upper frame, wherein theupper frame supports the at least two robotic arms, the robotic armsincluding at least one seeker subassembly and at least one pickersubassembly both in communication with the computer system, the seekersubassembly including at least one sensor, configured to sendinformation regarding potential agricultural targets to the computingsystem for analysis, the picker subassembly including a vacuum assemblycoupled to a nozzle with a terminating end, wherein the terminatingnozzle end includes a segmented ported section; the picker subassemblyfurther including two grappling spoons, the grappling spoons configuredto pinch together toward the vacuum nozzle. In some examples,alternatively or additionally, the picker subassembly vacuum nozzle ismounted on an extender actuator. In some examples, alternatively oradditionally, the baffle section is made of resilient, pliable material.In some examples, alternatively or additionally, the at least one sensoris at least one of a camera, laser, and lidar. In some examples,alternatively or additionally, a foliage management system includes twosets of belt rollers in communication with the computing system, twobelts, around each of the respective belt roller sets forming an insideand an outside, the belt roller sets and belts configured on thetraversing machine to receive foliage between the two belts, on therespective insides of each, wherein the belt roller sets aresynchronized with the at least two wheels or tracks such that a relativemotion of an inside of the belts and a surface on which the traversingmachine is moving, is approximately zero. In some examples,alternatively or additionally, the foliage management system includes anambient air vacuum pump in communication with the computing system, thevacuum pump configured above the two sets of belt rollers configured tosuck air away from plant beds over which the traversing machine passes.

BRIEF DESCRIPTION OF THE DRAWINGS

For a fuller understanding of the nature and objects of the disclosure,reference should be made to the following detailed description, taken inconnection with the accompanying drawings, in which:

FIGS. 1A and 1B are diagrams showing example mobile vehicle examples asdescribed in the embodiments disclosed herein.

FIG. 2 is a diagram showing example picker head example details asdescribed in the embodiments disclosed herein.

FIGS. 3A, 3B and 3C are diagrams showing example extraction hardwareexample steps as described in the embodiments disclosed herein.

FIG. 4 is a diagram showing example rotating picker head examples asdescribed in the embodiments disclosed herein.

FIG. 5 is a diagram showing more example foliage management features asdescribed in the embodiments disclosed herein.

FIG. 6 is a diagram of an example foliage management system as describedin the embodiments disclosed herein.

FIG. 7 is a diagram showing more example foliage management features asdescribed in the embodiments disclosed herein.

FIG. 8 is a diagram showing example conveyor belt examples as describedin the embodiments disclosed herein.

FIG. 9 is a diagram showing additional example conveyor belt examples asdescribed in the embodiments disclosed herein.

FIG. 10 is a diagram showing example point-to-point examples asdescribed in the embodiments disclosed herein.

FIG. 11 is a diagram showing example pneumatic removal examples asdescribed in the embodiments disclosed herein.

FIG. 12 is a diagram showing example sensor examples as described in theembodiments disclosed herein.

FIG. 13 is a diagram showing an example networked system which may beused in the embodiments disclosed herein.

FIG. 14 is a diagram showing an example computing system which may beused in the embodiments disclosed herein.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea sufficient understanding of the subject matter presented herein. Butit will be apparent to one of ordinary skill in the art that the subjectmatter may be practiced without these specific details. Moreover, theparticular embodiments described herein are provided by way of exampleand should not be used to limit the scope of the invention to theseparticular embodiments. In other instances, well-known data structures,timing protocols, software operations, procedures, and components havenot been described in detail so as not to unnecessarily obscure aspectsof the embodiments of the invention.

Overview

The systems and methods described here include anautomated/semi-automated system with machine(s) that is/are capable ofharvesting agricultural targets such as berries with robotic assembliestouching the plants or targets themselves. Example overall systems mayinclude subcomponents such as a seeker or sensor subsystem to find andlocate the targets, that works with and informs a picking subsystem toharvest the targets. The overall system(s) may be mounted on wheels ortracks to advance down a row of targets such as agricultural produce sothat the seeker subassembly may identify and map the targets while thepicker subsystem is used to harvest, gather, and move the targets withrobotic pickers.

In some example embodiments, additionally or alternatively, the seekersubassembly includes a camera or multi-camera system (such as astereoscopic arrangement) used by a remote operator to locate targetsand three dimensionally map them. In such examples, these mappedcoordinates may then be queued for harvesting. Additionally oralternatively, in some examples, the harvester subassembly is then ableto follow the seeker subassembly and harvest the berries whose mappedlocations are queued by the seeker subassembly. In some exampleembodiments, the harvester subassembly includes at least one robotic armwith multiple degrees of freedom capable of reaching into the foliage ofa plant and extracting a target such as a berry. In some examples, theextraction is aided by a vacuum system to hold the targets. In someexamples, additionally or alternatively, the extractions is augmented byone or multiple padded spoon graspers, capable of twisting and snappinga berry stem.

It should be noted that the examples used here describing berryharvesting, or even strawberry harvesting in the written descriptionand/or figures is not intended to be limiting and are merely used asexamples. The agricultural targets to which the systems here mayidentify, map, and ultimately harvest may be any sort including but notlimited to berries such as strawberries, blackberries, blueberries, andraspberries, other examples include grapes, figs, kiwi, dragon fruit, orother fruits. Vegetables may be harvested as well, such as Brusselsprouts, tomatoes, peppers, beans, peas, broccoli, cauliflower, or othervegetable. Any type of agricultural target may be harvested using thesystems described herein. Additionally or alternatively, the systems andmethods here may be used to target and gather non-agricultural itemssuch as garbage, or be used to take scientific samples such as rocks orminerals in environments or situations where it may be advantageous toavoid human contact or interaction.

Harvester Subassembly Examples

In some example embodiments, a harvesting subassembly is included as itsown separate vehicle system from the seeker/sensor subassembly.Additionally or alternatively, in some examples, the harvestingsubassembly may be in communication with or connected to the seekersubassembly. In some examples, seeker/sensor subcomponents areintegrated into the harvesting assembly and one machine incorporates allof the features described herein. The harvesting subassembly may includeany number of features that allow for autonomous, semi-autonomous, orhuman operable harvesting of delicate target agriculture such asberries, as described herein.

In some examples, either harvesting or seeker/sensor subassembly may bemounted on its own vehicle subassembly with wheels and/or tracks orcombination of both, to traverse down a row of agriculture with theseeker subassembly identifying and mapping the target berries and theharvester subassembly gathering targets.

FIGS. 1A and 1B shows figures of example overall traversing harvestingmachine to which any of the various subassemblies described herein maybe attached, mounted on, and/or coupled. Such a machine may be manned orunmanned, depending on the arrangement and level of automationprogrammed into it, as described herein. In the example, the maintraversing subassembly 152 includes various portions mounted to itincluding main driving wheels 154 and in some examples, guide wheels156. In some examples, guide wheels 156 may be canted outward in orderto support traversing a raised mound 101 should a mound be configured asshown. In some examples, tank treads or tracks may be used instead ofwheels 154, 156, and/or a combination of wheels and tracks may be used.

In some examples, sensors may be mounted to the frame 153 and/or chassis155 of the machine 152 as well as or in alternate of mounting them to arobotic arm. Such sensors may collect sensor data and send it to acomputing system on the harvester assembly or offboard to a back endsystem for processing and target identification. Such targetidentification may include coordinates of targets to be picked, andthose coordinates sent back to the system for robotic picking asdescribed herein.

In some examples, any number of robotic arms 160 may be mounted to anyof various frame portions 153 and/or chassis portions 155 that comprisethe overall traversing subassembly 152. Such robotic arms 160 mayinclude picker assemblies, sensors, or combinations of both. It shouldbe noted that many variations of robotic arms 160 may be used in thesystems described here, including but not limited to robotic wrists withlink and joint combinations with linear and rotational links, gantryrobots with linear joints, cylindrical robots connected to rotary basejoints, polar robots for twisting, and/or jointed-arm or articulatingrobots with twisting joints and rotary joints. Any combination of theseor other robotic assemblies 160 may be used on the systems describedherein to manipulate a picker head and/or sensors for harvestingagricultural targets as described.

For example purposes, the range 195 of the robotic arm 160 is shown inthe FIGS. 1A and 1B to show that the robotic arm 160 may reach differentsides of the row mound 101 where targets may be found and an accumulatorfor processing targets, such as the example traversing conveyor 178. Inexample embodiments, the robotic arm 160 may include various numbers ofjoints thereby allowing for various degrees of freedom to move aroundand about the plants and rows, taking different angles for the pickerassembly on the robotic arm 160 to extract a target. In some examples,the robotic arm 160 may include six degrees of freedom. In someexamples, the robotic arm 160 may include seven degrees of freedom, orany other number. In various example embodiments, the robotic arm 160may be any of various lengths, thereby affecting the range 195 of thearm, which may be tailored to the needs of the particular field or moundor target. In some examples, the robotic arm 160 may include one or moretelescoping portions, which may be elongated and/or retracted, therebyaffecting the length of that portion and the overall reach 195 of therobotic arm 160.

In some example embodiments, the robotic arms 160 may be ruggedized inthat the tolerances and durability of the arms are developed foroutside, dirty employment. In such examples, the robotic arms are not tobe operated in pristine factory settings. The systems described here mayoperate in weather, precipitation, dirt, mud, heat, cold, and injarring, rough conditions. As such, the bearings, tolerances, andactuators may be made of more durable materials than factory roboticassemblies. In some examples, extra gaskets may be fitted into thevarious joints to keep dirt out of the more delicate metal couplings andpivoting features of the robotic arms. In such examples, gaskets may bemade of rubber, plastic, or ceramic. The robotic arms may be made withfewer joints to minimize the number of potential problems that mayoccur. The robotic arms may be made of thicker materials, may beheavier, and be rust-proofed, waterproof, weatherized, and/or otherwisereinforced.

It should be noted that the system in FIG. 1A, 1B is shown straddlingone row of plants. In some examples, one system may straddle two, three,or wider mounds of plants and the example in FIG. 1A, 1B is merelyintended to be an example, and not limiting. By making the system widerto straddle a second row, two sets of arms 160 may be used to pick tworows, or three, or four, or whichever number.

In some examples, multiple robotic arms 160 may fit onto one overalltraversing vehicle 152. For example, systems may include a primarypicker assembly with a clean-up/redundant picker assembly which operatesbehind the primary setup. In those examples, up to eight picker arms maybe employed, four on the primary and four on the clean-up assembly, withone or two arms operating on each side of two rows. The clean-up systemmay operate in the same way that the primary system operates, to findtargets that the primary system did not harvest, and/or to operate as aredundancy should one or more arms on the primary system malfunction.

In examples where targets are fruit plants which are harvested manymultiple times during a single growing season, often multiple times perweek, leaving fruit on a fruit plant may curtail the productivity of theplant. If the plant senses that it still has fruit on it, it may notproduce more fruit. This would limit production, so the goal is toremove all of the fruit when ripe. As the bed rows may allow for somefruit to drape over the side of the plastic, and become easily exposedto viewing, other fruit may grow under the foliage, or on top of the bedrow tops and be obscured by foliage. Therefore, to find and harvest asmuch fruit from each plant as possible, it may be necessary to maneuverthe foliage to better view and/or harvest fruit targets.

In some examples, foliage moving arms may be included on the roboticarms 160 to alter, move, displace, and or otherwise gently maneuver thefoliage of the plant to better expose the targets such as fruit berriesto be picked. In such examples, a bar, or arm, may be pulled across thetop of the foliage in order to temporarily move it out of the way forthe seeker cameras and/or the harvesting assembly to locate and grapplethe target. In some examples, this foliage moving arm on the robotic arm160 may be maneuvered parallel or substantially parallel to the top ofthe row bed, and pull across the top of the foliage, bending the plant,but not breaking the plant leaves. This may reveal targets under thefoliage, those laying on the top of the row bed, or those caught up inthe foliage.

In some examples, a flexible curtain may be dragged over the foliage, toavoid damage to the foliage, but still pull it out of the way for theseeker and/or harvester to operate. In some examples, this flexiblecurtain may be a plastic skirt, in some examples, it may be a fringed orsliced skirt. In some examples, it may have fringes that drape over thefoliage, and yet flex around the foliage so as not to damage it. As theflexible skirt is pulled over the plants, it helps the seekersubassembly find the targets more easily by limiting the area to betargeted with a clean backdrop. The flexible skirt may be dragged fromone side in one direction during a first harvest and the next time theother direction, to avoid biasing or pulling the foliage in the samedirection each time.

In some examples, the overall traversing subassembly 152 may include atransfer conveyor 178. Such a conveyor may include any number ofconveyor belts, chains, rope, or other mechanism that can pull materialsfrom one place to another. Such transfer conveyor may be used to collectharvested targets and move them to a packaging subassembly, or storageunit.

In some examples, either or both the picking and/or harvestingsubassemblies or overall harvester 152 may include location sensing anddetermining devices. In some examples, GPS location sensors may beconfigured on both or either subassemblies 152 for the computing systemsto determine locations and/or steer as discussed in FIG. 13. In someexamples, cellular towers and signals may be used for location sensingby the subassemblies. In some examples, inertial navigations systems maybe used such as a ring laser gyro, a magnetic gyro and/or any othercombination of such with a computing system. In some examples,additionally or alternatively, the cameras in the seeker subassemblyand/or other cameras on the harvesting subassembly may be used toidentify and track an agricultural row 101 down which the vehicle 152may be steered. The location sensing and/or steering may be fed into anycomputing system, either located on the harvesting/seeking systems orremotely, in order to autonomously, semi-autonomously and/or allow forhuman activated remote steering. Any combination of these or othersystems may be used to locate and/or steer the systems here.

In some examples, the systems and methods may utilize self-steering onrow 101 with the intention to utilize a human user droid tender to steerthe droid off-row for unloading accumulated berry containers andreloading empty containers, then finally steering the droid back onto anew row to be picked. The droid may have the ability to be converted tofull autonomous mode for turnaround at the head lands as well asunloading and loading target containers.

Picker Head Examples—Vacuum Point of Contact

In some example embodiments, the harvesting subassembly may include atleast one picker head that first interacts with the target in the fieldto remove or detach the target from the plant it grows on. Such pickerheads may be affixed to or be part of, attached to, or otherwise incommunication with the robotic arms 160 as discussed in FIG. 1A, 1B. Insome example embodiments, at least one picker head may be mounted on theend of a robotic harvesting arm, alone or in combination with a sensorand/or lighting system.

FIG. 2 shows an example picker head assembly, with a front 2A and side2B view of the same assembly in detail. In the example shown, the mainpicker head assembly 202 is mounted with two actuators, one actuator fora pincers 204 and one actuator for an extender 206. In some examples, noextender actuator 206 is utilized. In examples where an extender isutilized, the extender actuator 206 moves in and down to move the mainvacuum nozzle 203 up and down, relative to the robotic arm (not shown).

The main nozzle 203 may be a hollow tube which may include a vacuum pumpassembly at one end (not shown) and the other end may be used to securea coupling suction portion 230 to a target 250 such as a berry. In someexamples, the coupling portion 230 may include one or more bellows orbellow configurations 232 that allow the coupling portion 230 to stayflexible and malleable to couple with the target 250. The compressionnozzle portion 230 may include a malleable hood or coupling section 232which may include one or more bellow sections, and a rim 234 around anopening 236 to aid in coupling to a target. In some examples, thecompression coupling portion 230 is or made up of at least one of, orcombination of a neoprene sleeve, a silicone sleeve, a rubber sleeve, orother natural or synthetic material that is soft and flexible. Such amalleable coupling section 232 may be configured to deform or otherwisecompress when a target 250 is contacted and may include baffles or otherstructure that allows for deformation and malleability. Such adeformation or compression may allow for the rim 234 to more easilyconform to the target 250 and thereby form a better suction fit for theopening 236.

In some examples, the compression coupling portion 230 may be 1.250inches in diameter, in some examples, the nozzle may be between 1.000and 0.750 inches in diameter, in some examples, the compression couplingmay be 2 inches in diameter. In some examples, the coupling portion 230may have an effective ported area of 0.44 sq. in. at rest and anexpanded area when stretched over the apex of a berry of 1.56 sq. in.But in any case, the nozzle could be customized to any size of intendedtarget.

In some examples, the compression nozzle portion 230 may include aninternal reverse conical mesh to help capture the target 250 yet be asgentle as possible on them. In such examples, the mesh creates anenvironment where the negative vacuum is acting on a broader surface ofthe target, thus minimizing the chance of target damage from localizedcontact to the grappler edges. This mesh may form a cradle for thetarget to lay in even while being picked, handled, and moved. Such amesh can be made of silicone materials for durability and flexibility.Alternate materials may be used such as a wire mesh, a plastic mesh, ora combination of wire mesh with plastic coating. Silicon coating may beused on a wire mesh in some example embodiments as well. In someexamples, the compression nozzle portion 230 and the opening 234 may besized for a most average target 250, big enough for the biggest targetsand flexible, but able to grasp and vacuum even a smaller target.

Examples may also include an internal spring system, inside orintegrated into the coupling portion 230. Such a spring system may bemade of plastic or metal coil(s) that help return the coupling portion230 back to an extended shape after a target is released by turning offthe vacuum and thereby deposited. Additionally or alternately, an irisor camera lens feature may be integrated into the nozzle 230. In suchexamples, the system may be able to adjust the size of the opening ornozzle end for different sized targets, opening for larger targets, andconstricting for smaller targets. In such examples, a coil or springcould be wound tighter for smaller targets and wound looser for largertargets.

In some examples, the vacuum hose 203 may be connected with the mainnozzle 230 to impart a suction or lower than ambient pressure within thenozzle tube 203, and thereby be able to attach to and secure a target250. In some examples, a vacuum subsystem with a vacuum pump may bemounted on the harvesting subassembly and a vacuum hose may run throughor around each harvesting picker robotic arm. In some examples, vacuumsubassemblies may be mounted on the robotic arm itself, along with avacuum hose on the picker head 202.

In some examples, the amount of suction power that the pneumatic vacuumsystem/pump may impart through the tube 203, may be 35 inches ofnegative vacuum. In some examples, 50 inches of negative vacuum may beused. Alternatively or additionally, in some examples, less than 80inches negative of vacuum may be used so as to avoid damage to thetarget 250. Alternatively or additionally, in some examples, the amountof suction power may be between 35-50 inches of negative vacuum orbetween 50-70 inches of negative vacuum may be used. Additionally oralternatively, the vacuum system may be able to reverse from suction toblowing air outward, to clear debris from the bellows, before switchingback to a suction mode for harvesting.

In some examples, a picker head 202 may include two grappler spoons,prongs, extenders, arms, or otherwise structures for grasping a target212, 214. In some examples, three spoons may be employed in a similarmanner as those examples shown with two as in FIG. 2. In some examples,four grappler spoons may be configured in two axes around the pickerhead 202 assembly. In some examples, alternatively or additionally, thegrappler spoons include a hinged and/or spring loaded portion at the endto better cushion the target 250 when pinched. In some examples, thegrappler spoons 212, 214 may pivot about the nozzle 203 to impart atwisting motion to snap a berry or other stem as discussed herein.

Another portion of the example embodiment of FIG. 2 is the grapplerspoons 212, 214. The grappler spoons 212, 214 may be configured with themain nozzle 203 between them and be configured to move in a pincermotion toward the nozzle 203 by a robotic actuator 216 and a hinge 218arrangement as discussed in more detail in FIGS. 3A, 3B and 3C. In someexample embodiments, the grappler spoons include a cushion 220, 222. Insome examples, the cushion 220, 222 may be made of or include closedcell foam, neoprene, gel filled pads, liquid filled pads, open cellfoam, layers of foam of different densities, a foam backing with a gelfilled pad on top, and/or any combination of the above or other materialthat may cushion a target 250 when the grappler spoons 212, 214 pinchthe target 250. In some examples, the material contacting the target 250is no more than 20-30 durometer in hardness.

In some examples, a pneumatic trash cleaning air jet 224 may be mountedto the end of the grappler spoon 212, 214 in order to help clear debris.In such examples, air holes may be configured on the end lip of thespoons and face in various directions to direct air toward foliage. Insome examples, a line of holes may be configured on the end lip of eachgrappler spoon 212, 214.

Example Picker Head and Picking Steps

FIGS. 3A, 3B and 3C show three example snapshots in the multi-stepprocess of target 350 acquisition and grappling using the picker headassembly 302 as described, where each step shows two angles 313, 305 ofthe same picker head assembly 302, front 313 and side 305. In an exampletarget acquisition, first, 3A, the picker head 302 is directed to atarget 350 by a seeker subassembly as discussed herein, using passedcoordinates and/or manually steered. Once directed and in place, arobotic arm (860 in FIG. 8) maneuvers the picker head 302 into closeproximity of the target 350 where the compression nozzle portion end 330may attach to the target 350 (as described in FIG. 2) resting on theground or surface 301 using low pressure imparted by the vacuum nozzle303. In some examples, this vacuum nozzle 303 may be maneuvered in anextended configuration 344 using the extension actuator 306. In such aconfiguration, the compression coupling portion 330 may attach, suction,or otherwise temporarily hold the target 350 and thereby secure thetarget 350 with the vacuum suction through the vacuum tube 303.

In examples using a pneumatic vacuum, it may be used to first secure thepicker head assembly 302 to a target 350 such as a strawberry. Thisvacuum attachment 303 to a target 350 may allow for the picker assembly302 to extract a target 350 from foliage, pick it off a stem and/or offa resting surface 301. By securing a vacuum attachment 330 to a target350 first, further even more grappling, twisting, and/or handling, maybe accomplished with spoons 312, 314, as described herein to aid inharvesting and moving targets.

In such examples, pliable bellows 330 may be positioned on the roboticassembly to within range of a specified target 350 and the bellowssection 330 extended 344 to within vacuum range. Through the bellowstube 303, a negative pressure may be generated by a pneumatic vacuum,pump, or other air or pneumatic suction device (not shown) therebysucking air through the nozzle end 336, through the bellows tube 303 andthereby through the various ports, holes, slots, and/or other shapedvoids in the end of the bellows. As such bellows 330 material may bemade of pliable material to be able to better conform to the shape of atarget more easily to thereby secure a better vacuum hold on the target.

Such a task is made more difficult by the variety of shapes, sizes, andorientation of targets needed to be grappled. Such a task is also mademore difficult by the potential of nearby foliage, other targets, stems,dirt, sticks, etc. which could interfere with the vacuum suction, and/orreduce the vacuum pressure that may be applied to a target by thebellows. The purpose of the bellows vacuum system is to create enough ofa vacuum attachment to the target such as a strawberry to secure thetarget from all or as many orientations as possible, including the apexpoint in examples of a target such as a strawberry.

In FIG. 3B, the same picker head assembly 302 is shown with a front 313and side 305, the main nozzle tube 303 may be retracted using theactuator for the extender 306. In some examples, the actuators 306, 304may be pistons operated by pneumatic and/or hydraulic features incommunication with a computing system to operate them. In some examples,the extender actuator 306 may operate in a generally upward/downwardmotion 340 away from the ground 301 or surface and toward the interiorof the picker head assembly 302. Some examples may forego the retractionstep and not utilize the extension actuator 306 and/or may not beconfigured with one. In examples where retraction is utilized, with thetarget 350 attached to the compression nozzle portion 330 which is nowretracted into the picker head assembly 302, the target 350 may begenerally aligned with however many grappler spoons 312, 314, forexample, are fitted on the picker head assembly 302. In some examples,the compression nozzle portion 330 and thereby the target 350 may beretracted to align with the respective spoon cushion portions 320, 322of the grappler spoons 312, 314, no matter how many grappler spoons areutilized.

In some examples, as shown through the FIGS. 3A, 3B and 3C, the spoonactuator 304 may be a piston assembly in communication with a computerto receive instructions and send data and configured to raise and lowera bracket assembly 318 on the picker assembly 302 which may interactwith a top end 372, 374 of each spoon arm 312, 314 to pivot each spoonarm 312, 314 about a pivot axis 310, 311, thereby opening and closing,or pinching the two spoons 320, 322 together, and moving them apart. Insome examples, the spoon arms 312, 314 may include springs in the pivotaxis areas 310, 311, which may bias the spoons in the closed or pinchedposition, and the bracket 318 may move in relation to the top spoon ends372, 374 to move down to work against the spring tension to open thespoons, and up to allow the springs to pinch the spoons 312, 314together. This actuation may take place due to the interaction betweenramped or angled portions of the top of the spoon arms 372, 374, and thebracket 318 moving against the spring tension in the pivot axes 310,311, pivoting each arm 312, 314 about its respective axes 310, 311.

As targets 350 may vary in size and shape, the alignment with the spoons312, 314, may be obtained by retracting 340 the compression nozzleportion 330 so that the rim 334 of the compression nozzle portion 330 isat a place just above the respective cushion portions 320, 322 of thegrappler spoons 312, 314 thereby ensuring that the respective cushionportions 320, 322 of the grappler spoons 312, 314 are able to pinchtogether to grasp 342 the target 350 without touching the compressionnozzle portion 330 when they are in the closed position. Other shapesand sizes of spoon pads 320, 322 may be used to cushion the targets 350when the pincher arms 312, 314 are brought together.

In some examples, sensors may be placed on or near the grappler spoons312, 314 and/or cushions 320, 322, at the hinges 310, 311, under thepads 320, 322, or other areas, to sense the size and/or shape of thetarget 350 once gripped. In such examples, a feedback loop may be usedfrom the sensor data to adjust the distance the compression nozzleportion 330 is retracted to align the target 350 with the grapplerspoons 312, 314. In some examples, such a sensor may be a light sensor,a laser sensor, a proximity sensor, piezoelectric pressure sensors,and/or a camera to align the target 350 with the grappler spoons 312,314.

In some examples, a pneumatic trash cleaning air jet 324 may be mountedto the end of the grappler spoon 312, 314 in order to help clear debrisin the field, foliage, and other obstructions in the field to aid intarget 350 acquisition. Such an air jet 324 may include one or morenozzles attached to pneumatic pumps and tubes that are able to blastjets of air in various directions, thereby moving, flapping, orotherwise disturbing plants, leaves, dirt, sticks, stems, or otherdebris that the system is not trying to target, but might be in the wayof a target. In some examples, the ends of the grappler spoons 312, 314themselves may include one or more nozzles, ports, or holes for air jetsto blast debris. In some examples, the outsides of the grappler spoons312, 314, opposite the respective cushion portions 320, 322 may includeone or more nozzles or ports, or holes for air jets to blast debris.

In the example of the third step, FIG. 3C showing the same picker headassembly 302, front 313 and side 305, when the compression nozzleportion 330 and thereby the suctioned target 350 is retracted 340 offthe ground or surface 301 and aligned with the grappler spoon cushions320, 322, the gripper spoons 312, 314 may be actuated by the grapplerspoon actuator 304 as described, and squeeze together 342 to grasp thetarget 350. In some examples, the retractable end of the baffler 332 mayretract above or past the point where the pincher spoons 312, 314 maypinch together 342 so as to be able to hold a target 350 with thesuction through the pliable baffles section 332 and the spoons 320, 322at the same time. Only by retracting 340 far enough could a target 350be secured by both systems at the same time. Such a configuration alsoallows for a handoff, for example, the suction may be turned off oncethe retracted portion 332 is secured by the pincher spoons 320, 322.

In this third configuration 3C, the target 350 may be grasped by thegrappler spoons 312, 314, and may still be held by the compressionnozzle portion 330 and the vacuum pressure from the main nozzle 303. Insuch examples, a feedback loop to the onboard or offboard computingsystems may be used from the sensor data to adjust the pressure used tosqueeze the target 350. In some examples, additionally or alternatively,a single expansion spring (not shown) that may be connected betweenspoons 312 and 314 may be used. The spring may set the tension that thegrappling spoons 312 and 314 may exert onto the target 350. In someexamples, additionally or alternatively, the sensors may bepiezoelectric pressure sensors on or under the grappler spoon cushions320, 322. In some examples, additionally or alternatively, the sensorsmay include cameras to visually detect securing the target 350. In someexamples, additionally or alternatively, the sensors may be tensionsensors on the grappler spoon actuator 304 to sense the pressure exertedon the closure of the grappler spoons 312, 314. In some examples,additionally or alternatively, the sensors may be tension sensors on thegripper spoons 312, 314 and/or in a hinged portion of the gripper spoons312, 314 to sense the pressure exerted on the closure of the grapplerspoons 312, 314. In some examples, feedback loops may be analyzed bycomputer systems in communication with the grappler spoon sensors 312,314, and also in communication with the actuators for the grapplerspoons 312, 314 to adjust the pressure on the target 350, which may beused to secure differently sized targets 350 while minimizing damage tolarger targets 350 and/or making sure smaller targets 350 are secured.

In some example embodiments, once the gripper spoons 312, 314 havesecured the target 350, the vacuum suction may be turned off by thecomputer, reduced, or otherwise cut off from the compression nozzleportion 330 which would in turn release the pressure holding the target350 to the compression nozzle portion 330 but leaving the target 350 incontrol of the grappler spoons 312, 314. In some examples, the paddedgrappler spoons 312, 314 may be configured to rotate 333, to therebyflick and turn the target 350 to remove them from their stems andthereby avoid having to cut a stem or plant in any way. This removalprocess of a target 350 from a stem may be advantageous in the shelflife of the target after harvesting and may be cheaper and easier toaccomplish in the field.

In some examples, this twisting motion 333 may be a 90 degree twist ofthe grappler picker head assembly 302. In some examples, this may be a180 degree twist. In some examples, this may be between an 80 degree and100 degree twist to snap a target 350 stem. In some examples, a snippingelement may be used in lieu of or in addition to the snapping, twistingmotion of the grappling spoons 312, 314. In such examples, a longertarget stem may be desired, and snapping or twisting may remove the stemclose to the target 350.

In some examples, the snapping of the stem by twisting may benefit ifthe stem of the target 350 plant is pulled out and away from the plantin order to impart a strain on the stem. In such examples, the pullingof the stem first, and then twisting the stem may result in cleanerand/or more accurate stem snaps. In some example embodiments, theextension/retraction actuator 306 may include a sensor that may senseresistance as the target 350 is retracted. In some examples, tensionsensors may be placed in joints of the robotic arm(s) in communicationwith the computing systems, to make such a determination.

In some examples, the twisting motion 333 may be imparted only when theresistance of the retraction of the target 350 meets a particularthreshold, as determined by computing systems in communication with suchsensors, thereby indicating that the stem of the target 350 is understrain or is otherwise stretched. Such resistance sensors may include apiezoelectric sensor, a strain gage, or other sensor mounted in or onthe retraction actuator 306. In some examples, the target may be a berrythat includes a calyx portion where a few leaves and the stem attach tothe target berry. In some examples, this calyx portion may be identifiedby the seeker subassembly to help determine which direction to pull thetarget, normal to the calyx portion. In such examples, the camera andcomputer system may be able to identify a color variation between theberry itself and the calyx leaves and thereby the stem.

In snipping examples, a scissors, saw, clipper, or other sharp pinchermay be secured to the picker head assembly 3C to cut the stem of thetarget 350 at a desired length.

Multi Headed Picker Examples

In some examples, additionally or alternatively, one robotic arm mayinclude more than one picker head. Such an arrangement may allow forfaster picking, as picker heads move between harvesting and droppingtargets to be packaged and processed.

FIG. 4 shows an example of a multi-picker, in this example, a threeheaded picker rotating assembly mounted on one arm 460 which may beconfigured to harvest and drop targets into a bucket accumulator. In theexample of FIG. 4, each robotic arm 460 includes three picker heads402A, 402B, 402C. The picker heads 402A, 402B, 402C may be configured asdescribed in any of the various example embodiments described herein. Insome examples, the picker heads 402 may be variants of one another, forexample, different sizes that are configured to harvest a certain sizedtarget. For example, a small, medium, and large picker head 402A, 402B,402C may be configured on one robotic arm 460 with different diameterson each nozzle or coupling portion.

As designed, and in use, when the targeting sub-system identifies atarget and estimates its size, it may assign the respectively sized oneof the three sized picker heads 402A, 402B, 402C to harvest that target.In some examples, the various picker heads 402A, 402B, 402C havedifferent features on them, for example, different air jet blasters fordifferent foliage situations. In some examples, the various picker heads402A, 402B, 402C may be redundant for maintenance and allow for one ormore heads 402A, 402B, 402C to break or malfunction, and then beautomatically replaced by a new and functioning picker head assembly402A, 402B, 402C. In some examples, the functionality of the pickerheads 402A, 402B, 402C may be triaged or placed in a hierarchy, so thebest, most functional picker head 402A, 402B, 402C is used, and if itbreaks further, then the next most capable head 402A, 402B, 402C isused, etc.

Any of various examples of multiple heads and uses for multiple headsmay be employed by the systems described herein. It should be noted thatthe number of three picker heads is not intended to be limiting and anynumber of picker heads may be mounted to a single robotic arm. In someexamples, each robotic arm 460 could include two picker heads 402A,402B. In some examples, each robotic arm 460 could include four or morepicker heads 402. In such examples, the picker heads 402A, 402B, 402Care configured onto a rotating portion of the robotic arm 460 to spinand/or rotate 499 to the target area and to a drop off or packing areaas described. In some examples the rotation is in a directionperpendicular to the ground, on an axis parallel to the ground as shownin FIG. 4. In some examples, the rotation may be in a carousel fashion,parallel to the ground, with an axis perpendicular to the ground. Theexample shown is not intended to be limiting.

In the example, first 4A, one of the multiple picker heads 402A firstcouples to a target 450. Next, 4B the picker head rotates 499 to bringthe first picker head 402A with the coupled target C50 into alignmentwith whichever accumulator 470 is being used. In some examples, thisrotation also brings another picker head assembly 402B into approximatealignment with another target C50. When in position, the first pickerhead 402A with the target 450 decouples, releases, or otherwise dropsoff the target 450 while the second picker head 402B couples to anothertarget 450 and the assembly can then spin or rotate again 499.

Foliage Management Examples

In some examples, in the field, the targets may grow and rest not onlyon the sides of the walls of the planter beds, but sometimes on the topsof the planter bed crowns and sometimes within the foliage of the plantitself. In such examples, the goal of finding and harvesting the targetsusing the systems and methods here may be more difficult than if thetargets are more easily presented. In some examples, the foliage may bedense, it may impede the sensors from finding targets. In some examples,alternatively or additionally, the foliage may impede the harvestingpickers from extracting targets, even if they could be found.

Thus, in such examples, it may be helpful to finding and harvesting moretargets, to be able to manipulate the foliage on the plant itself tobetter reveal any targets that may be growing and/or resting inside orunder the foliage.

In some example embodiments, as described, an additional feature may bea trash grinder. In some examples, impediments such as dead foliage,spoiled targets, or other non-targets may get in the way of the seekersubassemblies and/or harvesting subassemblies such as a picker head,from cleanly finding and harvesting targets. In such examples, thesystem may employ an additional trash disposal system.

FIG. 5 shows an example robotic assembly with an optional muncher,grinder, or other composting feature head with a target berry 550 andfoliage, trash, dead leaves, or other material 552 that is not to beharvested by can get in the way of the targets 550. Such trash 552 mayobscure the sensors from the targets to be harvested 550 and may clogthe vacuum or picker head assemblies as shown in FIG. 3A, 3B, etc.Therefore, a trash muncher features may be used to clear debris 552 thatis not desired to be harvested and can impede harvesting.

In such examples, additionally or alternatively mounted on and orseparately from the picker heads and seeker heads, a separate hose 510or hose used for the trash muncher features with suction capabilitiesincluding a separate or shared suction or vacuum pump (not pictured) mayinclude grinding blades 520 that are capable of spinning by way of amotor 560. The vacuum pump and/or motor 560 may be in communication withthe computer system as shown in FIG. 14 etc. to be able to turn spinningblades on, turn spinning blades off, speed up rotation, slow downrotation, turn suction on, turn suction off, etc.

In such a way, the trash material 552 may be suctioned up and be groundup to be collected or ejected 530. In some examples, such grinder blades520 may be located at the mouth of the vacuum tube 510 as shown. In someexamples, additionally or alternatively, blades 521 may be locatedfurther up the vacuum tube 510 instead of or in addition to blades 520at the mouth of the tube. A motor 560 may be used to spin the blades520, 521 inside the tube 510 and be in communication with he computer tosend and receive data including commands to turn on, off, speed up, lowdown, the spinning blades.

A pump or vacuum system (not shown) may be coupled to the vacuum tube510 to suck air, trash, and/or impediments up 522 and away from thesurface 501. Such air and impediments, after being ground by thespinning blades 520, and/or 521 may be sucked back and out 530 to theground, away from the plants, or to a collection bin (not shown). Such avacuum pump (not shown) may be in connection with a computing systemonboard to control the vacuum motor to start, stop, speed up, slow down,or any other kind of command. In such a way, a clean-up or preparationstep may be added in order to remove impediments to help expose targetsbefore the sensors look for and map targets, and picker heads harvestthe targets.

In some examples, additional trash grinding systems may work inconjunction with the sensor systems to identify impediments for removal.Such systems may utilize the same targeting and coordinate mappingsystem as the target systems, but instead employ the trash grindingsystem to remove the impediments instead of harvest the targets. In sucha way, an iterative process may be to utilize the sensors to firstlocate and map impediments for the trash grinding system to removeimpediments, then the sensors may be utilized to locate and map targetsfrom the cleaned up plants, to the pass to picker heads to harvest. Insuch a way, additional targets may be harvested that otherwise might beunseen or too hard to reach.

Foliage Roller and/or Pneumatic Examples

Additionally or alternatively, FIG. 6 shows an example system withfoliage management features to aid in finding and/or harvesting suchtargets. As shown in FIG. 6, three views of the same scene are shown, atop down view 601, a side view 603 and an end-on view 605 to the planterbed 660, harvester portions, and plants 620. It should be noted that inFIG. 6, not all of the harvesting assembly is shown, only a portionhaving to deal with foliage management, and specifically, foliagemanipulating rollers 610, 612, pneumatic air jets 630, 632, roboticassembly 662, and pneumatic vacuum 640 and although they appear in theexample FIG. 6 as not connected to anything, would be mounted on theframe of the harvester, and are shown in FIG. 6 for illustrativepurposes only. It should be noted that all of these features areoptional and may be employed on the harvesting system alone or in anycombination described here or otherwise. They are not reliant upon oneanother but may benefit from employment of the other features, meaningany combination of the above may be used, alone or together.

In some examples, an arrangement of belts and rollers may push or bendthe main foliage of a plant 620 to allow the robotic assembly 662including sensors to more easily observe and/or harvest the targets orberries 602. Such foliage manipulating rollers may be more gentle on aplant foliage 620 than just a bar or pusher, because the speed at whichthe belts move may be synchronized with the forward movement of theharvesting machine down a row (as shown in FIG. 1A, 1B) such that thereis imparted no or little shear force on the foliage 620, but insteadonly the intended pushing or bending force to gently move the foliage620 to the side.

In the example views shown in FIG. 6, the harvesting machine may includebelts 610, 612, spun by rollers 614, 615, 616, 617, 618, 619 thatinclude pins or cylinders capable of turning mechanically by a chain,gear, or motor, as shown in the top-down view 601 in communication withonboard and/or offboard computer systems. Between and among theserollers may be a belt, sheet, ribbon, or other material 610, 612 kepttaught or semi-taught around the rollers 614, 616, 618. The belt 610,612 may be made of fabric, plastic, woven fibers, rubber, or plasticcoated fabric. Such material may be of a thickness that is sturdy forfield work, yet gentle on the plant foliage 620.

In the example of FIG. 6, the rollers 614, 615, 616, 617, 618, 619 aregenerally arranged perpendicular to the ground 660, pointing uprightsuch that the outside of the belts 610, 612, rotate in the same generaldirection as the movement of the harvesting machine 650. In this way,the inside of the belts, where the belts touch the plant 620, is movingthe opposite way that the machine is moving 650 down a plant bed row. Insome examples, this rotation may be synchronized with the forwardmovement of the machine down a plant bed row, by computers which receivedata regarding the speed of the harvester, and thereby the speed of theplants 622 as they are traversed, and the rollers 614, 615, 616, 617,618, 619 to speed up or slow down at the same time and rate, such thatthe interior side of the belts 610, 612, are moving at or about the samespeed as the plants 620 as the machine traverses the planter rows 660,such that the relative speed between the interior of the belts 610, 612,and the plants 620 is zero, or close to zero. This allows the plants 622to be touched by the belts 610, 612 with as little disturbance aspossible, and without shearing, ripping, or otherwise damaging thefoliage 620, and allowing only the bending or pushing of the mainfoliage 620 to be effected on the plant by the main roller system.

In some examples, the belts 610, 612, and rollers 614, 615, 616, 617,618, 619 are configured to interact with the plants 620 by squeezing themain foliage into a narrow opening between the two belts 610, 612. Insuch a way, the targets or berries 602 may be more easily seen by and/orharvested by the robotic arms 662 as described herein. In some examples,only one belt 610 may be arranged to push aside the foliage on the plant620 in one direction. In some examples, instead of the two belts 610,612, being aligned so as to squeeze the plant foliage 620 at the sametime, they may be staggered, one after the other such that the foliageof the plant 620 may be pushed to one side by one belt 610, then to theother side by the other belt 612 instead of simultaneously. Anycombination of these pushers to one side, and/or squeezing may beutilized.

It should be noted that the arrangement of rollers 614, 615, 616, 617,618, 619 may be more or fewer in number than three per belt 610, 612. Insuch examples, only two rollers may be used to roll a belt, or more suchas four rollers may be used to roll a belt. In such examples, and/orwith a three roller example, the configuration of roller positions onthe machine may be changed such that the shape of the rolling belt maychange. In the example of FIG. 6, the three roller belt configurationincludes a generally wide triangular shape followed by a generallynarrow triangular shape. This allows for the plant 620, when the machineis moving, to be gathered at a wide end by the first, most widely spacedrollers 614, 615, and then pinched into a narrower position between thenext rollers 618, 619, and held there through the rest of the traverse,where the robotic arms identify and/or harvest targets 602, before beingreleased at the last rollers 616, 617 when in use.

It should be noted that side view 603 does not show both belts, but onlyone belt 612 in either a one belt or staggered belt embodiment, orwithout the second belt to show the inside of the embodiment forpurposes of explanation. As mentioned, any configuration of one, two,staggered, or aligned belts may be utilized as described herein.

Air Jet Examples

In some example embodiments, alone or in combination, air jets 630, 632could be configured to blow ambient air onto the foliage 620 and/or theplant beds 660 to remove trash, clear debris, move foliage 620 orotherwise aid the robotic assembly 662 in finding and harvesting targets602.

In such examples, compressors and/or pumps (not shown) may sendcompressed air down a tube (not shown) and through a nozzle or jet 630,632 to push or maneuver foliage 620 for sensors and/or harvestingfeatures 662 to better find and harvest targets 602. Such compressorsmay be positioned in, on, or around the harvesting machine. In someembodiments, alternatively or additionally, such tubes may run downrobotic arms 662 and thereby be able to be maneuvered in the same orsimilar fashion as the seekers and/or harvesting picker heads 662. Insome examples, such air jets 630, 632 may be configured next to,adjacent, and/or near the seeker and/or picker head to aid in findingand harvesting targets 602.

It should be noted that the number of air jets 630, 632, does not needto be two as shown, and in fact could be any number of air jets,configured to blow air on the sides of the planter beds 660 and/ortargets 602 or plants 620. In some examples, the air jets may utilizedthe same vacuum system as the vacuum 640 if such a system is used toblow the same air sucked from the top. In some examples, a filter may beused to filter the air sucked by the vacuum 640 to blow by the air jets630, 632. In some examples, the air jets 630, 632 may include nozzleswhich concentrate the air flow into a faster speed. In some examples,the air jets 630, 632 may include diffusers, or other features that pushair into directions which may be able to be aimed or moved by a motor incommunication with a computer.

More embodiments of Air Jets are described in more detail in FIG. 7. Inthe example, similar to FIG. 6, three sides of the same plant 720 andplant bed 760 are shown with a top-down 701, end-on 705 and side view703. In the drawing, the plant 702 is being deflected to the side by aroller setup 710 as described in FIG. 6, or some other kind of arm orwall and harvested by any number of robotic assemblies 762 as describedherein. In such examples, various air jets connected to air pumps (notshown) could be arranged on the harvester system to blow ambient aironto the plants 720 and/or plant beds 760 to allow for more easilyfinding and harvesting the targets 702 by the robotic assembly 762. Inthe example of FIG. 7, there are air jets on both sides of the plant 720with one set of air jets 732 arranged to layover the canopy of the plant720 and comb the foliage. In the example shown, three air jets 732 areso arranged to blow air at the top of the plant canopy 720 and help pushthe plant to one side. In such examples, any number of air jets with orwithout nozzles or diffusers may be used, for example, three are shownin FIG. 7 but one, two, three, four, five or more could be used for thispurpose. These air jets 732 are generally arranged toward the top of theplant canopy, aimed to push or deflect the plant 720 in a particulardirection. Either alone or in combination with a belt as shown in FIG.6, the purpose of deflecting the plant foliage 720 would be to movedebris and material out of the way for the robotic assembly 762 to findand/or harvest targets 702 as described herein.

In some examples as shown in FIG. 7, another air jet 730 or set of airjets alternatively or additionally, may be configured for anotherpurpose. In such examples, the second grouping of air jets 730 eitheralone or in combination with the others described herein, may be used ascross-air jets to blow targets off the top of the plant bed 760 and downonto the side to make it easier for the robotic assembly 762 to findand/or harvest targets. In the example of FIG. 7, the cross air jets 730are arranged lower than the foliage air jets 732 and instead arranged toblow air down and across the top of the plant bed 760. This may inducetargets 702 to fall or otherwise move onto the side of the plant bed 760to make it easier for the robotic assembly 762 to find and/or harvesttargets 702. The number of cross-air jets could be one as shown, ormore, such as two, three, four, five, or more. The arrangement ofcross-air jets 730 may be configured onto the harvesting machine in anyway, connected to an air pump to blow ambient air onto the plant bed 760and/or targets 702.

In some examples, alone or in combination, alternatively oradditionally, air jets 730, 732, may be configured as comb-air jets thatfollow an elliptical path through and around the plant 720. In suchexamples, the air jets 730, 732, may comb through the leaves and canopyof the plant 720, thus releasing targets. Any arrangement of air jets tomanipulate foliage, release targets, or otherwise blow away debris anddead foliage may be used. And any combination of nozzles and/ordiffusers may be arranged on the air jets as described herein to eitherfocus the air into a faster moving stream in a nozzle, or widen thestream of air in a diffuser. And number of sub nozzles may be configuredon the air jets as well, in order to control the direction of the airjet streams of air, such sub nozzles may include movable parts,controlled by a computer to direct air flow in a desired location. Airpumps pushing air through the nozzles may also be controlled bycomputers to turn on, turn off, pulse, or otherwise push air toward anintended direction and velocity and/or pressure.

Vacuum

Turning back to FIG. 6, in some examples, a high volume suction blower,or vacuum 640 may be attached near the belt system. In such examples,the vacuum system 640 may pull air up and out away from the plantfoliage 620. In some examples, the vacuum system 640 may come justbefore the belts 610, 612 such that the entirety of the foliage 620 maybe pulled up or urged upward before being squeezed between belts 610,612. In such examples, the vacuum 640 may also remove some dirt, debris,dead leaves, bugs, or other unwanted materials. In some examples, thematerial captured from the vacuum 640 may be blown aside, mulched, orotherwise recycled.

In some examples, air jets 630, 632 may be used in combination with thebelt systems 610. In some examples, the air jets 630, 632 may bestaggered to control air positioning at the plants 620, targets 602,and/or plant beds 660.

Target Accumulator Examples

In some examples, additionally or alternatively, the harvestersubassembly may include any number of target accumulators. In someexamples, a conveyor belt system may be used to move accumulated targetsfrom the robotic arm and/or vacuum hose to a storage area and/or apackaging area. In some examples, a bucket system may be used to gatherand/or move accumulated targets to a storage area and/or a packagingarea.

FIG. 8 shows an example single headed picker assembly which isconfigured to drop targets into a multi-conveyor belt accumulatorassembly from the top down, and from the side. In FIG. 8, the pickerhead 802 is shown mounted to a robotic arm 860 which may position thepicker head 802 in any of various poses in order to couple to a target850. In the example, the targets are on a mound 801 which the robot arm860 traverses from above.

In use, multi-conveyor system as shown in FIG. 8 is positioned by therobotic arm 860, to extract a target 850 and drop it, and or/place itonto a first capture conveyor belt 880. In some examples, this captureconveyor 880 is configured to move perpendicular to a mound row 801and/or otherwise away from the plants and out toward an area which ismore open. In some examples, this capture conveyor 880 is a flatconveyor, and wide enough to allow for the movement or rolling oftargets, such as four inches wide. In some examples, the speed of thiscapture conveyor belt 880 may be between 1 in to 5 inches per second. Inthe example, this capture conveyor 880 may be configured to move targets850 to a second transfer conveyor belt 878. In some examples, thetransfer conveyor 878 is configured to move targets in parallel with arow mound 801, out and away from the ground to be processed.

In some examples, the transfer conveyor 878 may include fins or othercompartmentalized portions to reduce rolling of targets 850 as they moveup the transfer conveyor 878 at an angle. In some examples, the transferconveyor 878 may move targets 850 to a packaging station or otherhandling area on a main vehicle, or off the main vehicle to anaccumulator or other receiver section as described herein. In someexamples, the transfer conveyor 878 may be more narrow than the captureconveyor 880, such as between 3 and 5 inches wide. In some examples, thetransfer conveyor 878 revolves at speeds of between 5 inches and 10inches per second.

In some examples, a brush or other skirt assembly 876 is configured tobridge the capture conveyor 880 and the mound 801.

FIG. 9 shows an example set of diagrams showing the conveyor beltarrangement from the side, in three steps of an example pick. In suchexamples, a single picker head 902 is attached in a rotatable manner tothe arm 960. In FIG. 9, the first diagram 9A shows the picker head 902grappling a target 950 as described herein. The capture conveyor beltassembly 980 is shown moving the targets away from the plant 999 and bedand toward another removal arrangement such as a transfer conveyor 978or other conveyor or bucket system.

9B shows the picker head assembly 902 retracting the target 950. 9Cshows the picker head assembly 902 rotating (arrow) on the arm 960 anddropping or placing the target 950 onto the capture conveyor 980 whichmoves the target to the transfer conveyor 978. In such examples, thepicker head 902 would then rotate back to another target for grapplingand extraction.

Some examples may employ human packers, that is, the harvesting systemmerely uses conveyor systems as shown in FIGS. 1A, 1B, 8, and 10 totransport the targets to a central point on the system for a human tohandle and pack into a box, clam shell plastic, or other shipping/salescontainer. But some example systems may utilize a more automated systemto pack targets. In such examples, similar subsystems to those describedfor seeking, targeting, and picking, may be employed to deposit targetsinto a container and arrange the targets in a manner that minimizesempty space and are arranged in a stable arrangement using picker heads.In such examples, a three dimensional mapping of not only the targetitself but the other targets which are deposited into a container may beused to maneuver newly placed targets in an arrangement that is stableand uses the space provided. A three dimensional puzzle results in eachcontainer, placed by the picker arms or a separate set of picker arms(placer arms) that would operate by picking targets from the conveyor,and depositing them in a container, using the same suction and/orpincher system as described in FIGS. 2, 3A, 3B and 3C. In some examples,the grappling spoons may not be necessary to pack targets as only thebaffle and nozzle vacuum system may be enough to maneuver them.

FIG. 10 shows an example which uses either a single transfer conveyor ora bucket instead of a multi-conveyor system. In FIG. 10, a single headedpicker assembly which is configured with the picker head 1002 to suctioncouple to a target 1050 as described herein 10A, then move the target10B to a bucket, or other accumulator 1070 where the picker head 1002may then deposit, drop, or otherwise place the target 1050 for furtherremoval or processing. In some examples, the accumulator 1070 is atransfer conveyor. In some examples, the accumulator 1070 includes apackage.

FIG. 11 shows an example which uses a vacuum hose instead of amulti-conveyor system. In FIG. 11, a single headed picker assembly whichis configured with the picker head 1102 to suction couple to a target1150 as described herein 11A, includes a second vacuum assembly 1190move the target 11B for further removal or processing. In some examples,the pneumatic assembly 1190 includes a vacuum transfer tube 1192 whichmoves the targets 1150 out for processing. In some examples of thepneumatic transfer system as shown in FIG. 11, no grappling spoons arenecessary if the vacuum assembly 1190 is positioned relatively below thepicker head 1102. In such examples, the primary picker head vacuum mayswitch off when it is retracted and positioned over the mouth 1194 ofthe vacuum assembly 1190 and the target 1150 drops into the vacuumassembly 1190 and transfer tube 1192 by pressure differential suction.

Seeker/Sensor Subassemblies

In some examples, the harvesting described herein is directed by aseeker subassembly that is able to identify targets for harvesting, passcoordinates for the targets to the picker subassembly for extraction.Such seeker subassemblies may include any number of cameras (visiblelight, thermal, UV or other), radars, lidars, lasers, acoustic locationfinders, GPS, inertial navigation systems, piezoelectric sensors, and/orany combination of these or other sensors to locate and identifytargets.

In some example embodiments, as discussed herein, the seeker subassemblymay include a camera and/or multiple cameras arranged so as to be ableto view the target foliage and thereby the target agriculture to beharvested. In some examples, multiple cameras may be arranged on theseeker subassembly such that images taken from the multiple cameras maybe processed by a computing system to create three dimensional (3-D)images using machine vision. In some examples, these images are made ofpixels and the computing systems is able to identify targets representedby pixels to be harvested and map the targets in three dimensions. Insome examples, the cameras may be configured to acquire multi-spectralor hyper-spectral imagery to enable the use of advanced analysisalgorithms for evaluating fruit health, quality and ripeness. In someexamples, the images gathered may include those of a thermal imagingsystem for evaluating the temperature of the berry to be harvested.These cameras may comprise of cooled or uncooled sensors generatingarea-scanned images of at least 640×480 pixels. Some embodiments mayutilize a single thermopile based sensor to provide an integratedtemperature measurement of the mean temperature of the berry.

In some examples, the target identification is automated by ArtificialIntelligence (AI) computing systems working in conjunction with a neuralnetwork data base and the camera systems. In some examples, the targetidentification is aided by a human who is analyzing a visualrepresentation of the image data sent by the camera(s). The humanoperator may utilize human logic to perform quality control on the berryquality, correcting errors of identification by the AI systems, and todetermine grade (i.e. #1 grade, #2 grade, spoiled, immature). Inaddition, the human operator may determine the best approach angle forthe berry grappler to capture the fruit. In some examples, theprocessing computer may provide visual guidance and suggestions to thehuman operator as to the quality of the berry using automated algorithmsexploiting the spectral and spatial analysis of the target. In someexamples, the human may be at a remote location, not on or necessarilynear the harvesting machine or field itself, but at a location networkedto the camera systems to analyze image data coming from the camera(s)and sent by wireless communications. In some examples, the wirelesscommunications are at least one of cellular, Private Wireless Network,Bluetooth, satellite, radio, and/or any other wireless method usingantennae and processing computers. In some examples, a private wirelessnetwork will consist of a fixed or portable base station that forms amesh network with other fixed or portable base stations to provideseamless complete communications coverage of the entire field in whichthe harvester is operating. In some examples, the fixed or portable basestation will connect to the Wide Area Network, or Internet, through adedicated back-haul connection to an Internet Service Provider (ISP).The backhaul connection may utilize any technology capable of carryingthe necessary bandwidth to execute the harvester mission includingconnections comprising of dedicated wireless point-to-point links,copper twisted-pair links, coaxial based communications or optical fibertechnologies.

In operation, the berry grappler, once it reaches a predeterminedoff-set distance from the mapped coordinates of the berry to be picked,control of the movement of the robotic arm is handed off to an internalguidance system that will lock onto the targeted berry and fine tune anydiscrepancies in the logged coordinates that may occur from the forwardmovement of the harvesting platform. The internal guidance system willutilize a neural network and/or artificial intelligence in conjunctionwith accumulated data gathered onto a Neuro Network to make decisions.

To deal with leaves, stems and trash obstructing access to the targetedberry, the berry grappler will have the ability to recognizeobstructions and perform a second operation to clear access to thetargeted berry. The berry grappler will be equipped with a trashdiverter located at its most forward tip of the grappler spoons. Thetrash diverter may be equipped with air jets, or foliage hook device, orrotating paddle device and other appropriate methods to displace theobstruction. In operation, the berry grappler trash diverter will movein a progressive diameter rotation around the targeted berry location,clearing the obstructions. This second operation will only take placewhen obstructions are viewed by the stereo cameras.

In some examples, the three dimensional image data processed by and sentfrom the camera(s) may allow for a virtual reality environment to becreated for a human user. In such examples, a virtual reality headset ordisplay may be utilized by a user, remote or close to the harvester, toidentify target agriculture and thereby send the target mappingcoordinates to the harvesting machine for harvesting.

In some examples, the remote human users may be allowed by the computerto determine an angle of attack for the picker head to harvest anagricultural target, using the sensor data. In some examples, that mayinclude image data such that the user may angle the picker head assemblyusing an interface such as a joystick, touchscreen, or mouse.

In some examples, data created by the cameras and data created by thehuman selection of agriculture is stored by a computing device. In suchexamples, the identification data may be amassed in order to analyze andlater create algorithms for neural network engines to process. In suchexamples, after much data of targeting, identification, and harvestinginformation is gathered, an neural network engine can be trained may beable to replicate some or all of the human targeting using the threedimensional maps.

Examples of cameras which may be used in the described systems includestereo vision with resolution of 1920×1080 and frame rates of 30 persecond. In some examples, different stereo vision camera systems may beutilized, with different resolution and frame rates, these being onlyexamples.

In some examples, a suite of these or other sensors could be placed onthe frame or chassis of the harvester, mounted still, or on motors toswivel or turn. In some examples, a suite of these or other sensorscould be placed at the end of a robotic arm such as those shown in FIG.12. In the example, the sensor 1203 is able to detect the target 1250 inwhichever manner the senor operates (light, heat, lidar, acoustic,radar, etc.). In some examples, multiple sensors are placed on a singlearm 1260. In some examples, multiple arms 1260 operate with their own ormultiple sensors 1203. In some examples, the sensors 1203 are mounted ona rotatable mount, able to move rotate in one, two, three, four, five,six, seven, or more degrees of freedom. In some examples, a robotic arm1260 includes sensors, picker heads, and/or multiple sensors and/orpicker heads, and/or a combination of sensors and/or picker heads. Insome examples, the sensors may be stereoscopic sensors, spaced apart butaimed at a similar focal point to provide data for the computing systemto create three dimensional models of targets and environment aroundtargets.

In some examples, a seeker subassembly vehicle may work independentlyfrom the harvester subassembly, and in some examples, the twosubassemblies are on the same traversing machine. In some examples, theharvesting subassembly has its own wheels and/or tracks or combinationof both, to traverse down a row of agriculture and harvest the mappedtargets it receives from the seeker subassembly. In some examples, theseeker subassembly vehicle and harvesting subassembly vehicle are ableto mate, connect, and/or otherwise work in concert by connection. Insome examples, this connection includes a wired connection to allow fortarget information to be passed from the seeker subassembly to theharvesting subassembly. In some examples, the two subassembliescommunicate wirelessly. In some examples, a combination of wired andwireless communication may be arranged.

Lighting

In some example embodiments, the seeker subassembly includes variousspecialized lighting which may be used to find and identify targets.Such lights may be configured on the ends of robotic arms, integratedinto robotic arms that include picker heads, or cameras. Examples areshown in FIGS. 1A and 1B. Such lights may be fixed onto othersub-assemblies on the seeker assembly and/or harvesting sub-assembly.

In some examples, such specialized lighting may be configured to emit acertain wavelength or spectrum of wavelengths such as but not limited tovisible light, infra-red light, and/or ultra-violet light. In someexamples, the lighting may be at a wavelength that excites items tofluoresce. In some example embodiments, light spectrum filters may beused by the cameras described herein to filter out or delete wavelengths of light that would otherwise block out any fluorescentproperties reflected or emitted by targets such as berries.

In some examples, the specialized lighting may be light emitting diodeswhich are tuned to emit light at a specific frequency. In some examples,that frequency may be a combination of 470 nm (blue) and 635 nm (red).In some examples, the lights may be LED lights. In some examples, thelights may be incandescent lights. In some examples, the lights may behalogen lights, fluorescent lights, metal-halide, neon, high-intensitydischarge lamps, or any permutation or combination of any of the above.

Mapping and Passing Target Coordinates

As described herein, in some example embodiments, additionally oralternatively, sensors onboard the harvesting systems such as machinevision camera and computing systems may be used to map targetagriculture in three dimensions and pass the coordinates to theharvester subassembly for harvesting. In some examples, these arestereoscopically arranged cameras, with similar field of views to createimage data that may be used by the computing systems to create threedimensional maps of targets. These mapping coordinates may be describedin a global coordinate system such as Universal Transverse Mercader(UTM), or a local coordinate system frame relative to the coordinatesystem defined by the three dimensional imaging system on the harvester.In some examples, an X,Y,Z coordinate system may be employed using ananchor point in the camera view and/or on the traversing machine itself.

The various sensors described herein including but not limited tovisible light cameras, infrared cameras, ultraviolet cameras, lidars,radars, lasers, or other sensors may be used to scan the produce plantsand identify targets. Using the automated, semi-automated, or manualselection processes and systems described herein, the systems couldgenerate coordinates for selected targets. These mapped targetcoordinates may then be queued in a buffer or database, for theharvester subassembly to harvest. In some examples, after one coordinateis added to the harvesting coordinate queue, more targets may be addedto the queue to be harvested in turn. In such examples, the targetingsubassembly, machine vision, and target mapping may occur without lag ordelay in the handoff from targeting to harvesting, and not be hamperedby the limitations of the harvesting subassembly itself. In such a way,in some examples additionally or alternatively, the targetingsubassembly may be mounted on a separate vehicle to travel at its ownspeed and send targeting mapped data to the harvesting subassembly bywireless communications. In some examples, the targeting subassembly maybe a part of the overall machine and connected to or in communicationwith the harvesting subassembly to pass the targeting mapped coordinatequeue by wired communications to the harvesting subassembly. In someexamples, a cloud or distributed computing resource may be utilized sothat the targeting queue may be relayed or sent to the harvestingsubassembly wirelessly.

In some examples, the mapping may be done early or before a harvestermachine may come down a row. Additionally or alternatively, in someexamples, mapping may be done just before harvesting, on the samemachine in some examples to minimize the variables of the berries and/orfoliage moving. Any time between target mapping and harvesting may beutilized, depending on the circumstances of the harvest.

In some examples, mapping information may be stored in a remote serveras described in FIG. 13, cloud server, or distributed system, for thepurpose of future analysis (post processing) of the imagery to evaluatethe condition of the plant. Post processing operations may include anevaluation of the plant for disease, nutrient deficiency, unripe berryinventory, and/or other plant damage. Data gathering and analysis on alltypes of agricultural specifics may be accomplished using the suite ofcameras and/or sensors on the systems described herein. For example,outputs of post processing operations may be utilized to selectivelyaddress in-field issues at a plant-local scale that may otherwiserequire broad remedies using traditional methods. Other outputs of postprocessing operations may generate statistical data related toobservations and measurements that are made while the harvester isoperating in the field that can be advantageous to the growers businessefforts.

Planter Subassembly Examples

In some example embodiments, a subassembly may be utilized which may beused to automate the bed preparation and planting of plants in the bedsas an additional or alternative embodiment than the harvesting roboticassemblies described in FIGS. 3A, 3B, 3C, etc. In some examples, such asubassembly may include an arm with a propane torch instead of or inaddition to a picker assembly. In some examples, the propane torchincludes a circular metal shape within which the torch is lit. In suchexamples, when the circularly shaped torch is brought within range of aplastic sheet covering a planting bed, the torch may burn a circularlyshaped hole in the plastic, but leave no residual flap or otherintegrity diminishing aspects to the plastic. In such examples, thesubassembly may be configured to space the holes for the plantings inthe plastic at predetermined distances from one another. In someexamples, the subassembly may be configured to place one, two, or threelines of holes on one bed row, depending on how many plants areconfigured to be placed on one bed row. In some examples, the holes maybe side-by-side. In some examples, the holes may be staggered oralternating.

In some examples, after the robotic arm(s) has burned holes in theplastic covering the bed rows, another arm may be utilized to insert aseedling plant into the dirt under the hole in the plastic. In suchexamples, camera arrangements may be utilized to identify the holes inthe plastic and align the robotic assembly to the center of the hole. Insuch examples, the robotic arm may then utilize a spade shaped tool todig into the dirt under the hole a predetermined distance to make roomfor roots of a seedling plant. In some examples, a spoon or pinchmechanism may be used to grasp and insert the seedling plant in the holedug by the spade, at a predetermined distance into the dirt, and insidethe hole in the plastic. In some examples, the hole may be made with thesame tool that plants the seedling. In some examples, this is a two-stepprocess, to make the hole and then plant the seedling. In some examples,this is a one-step process, to make the hole and plant the seedling.

Computer and Computer Network Examples

Additionally or alternatively, the systems described here may be used toharvest agricultural targets in an automated, semi-automated, or evenmanually controlled manner. In some examples, the semi-automated mannermay be arranged in a remote setting, allowing for a human to interactwith camera views from the harvester to help target the produce. Such anarrangement may be made possible by a network and computer arrangementshown in FIG. 13 and/or FIG. 14.

The variations on these options depend on how much a remote or localcomputing system may be programmed to identify and harvest a target. Forexample, in a fully manually controlled system, a human operator maycontrol the movements of both the seeker system and the harvestingsystem. In such examples, by remote control using a joystick or othercomputer driven operating device(s) a human could scan the rows ofplants for a target using the camera systems, and even maneuver therobotic arms that the camera systems are connected to, to identifytargets, and then use a control system such as a joystick to maneuverthe picker head assembly to the target, and then harvest the target asdescribed herein. Such examples would allow for remote operation of thesystems such as by wireless control to allow for human controllers to bestationed anywhere in the world, through some kind of wireless uplink.

The other extreme of control systems would be a fully automated system.In such a system, the traversing machines would move down a row ofagricultural targets and the seeker subassembly would use machinelearning/artificial intelligence/neural networks/and/or otherprogramming to seek out and identify targets with the seekersubassemblies and then harvest them as described using the picker heads.Such examples would depend on computer algorithms and programs todetermine using the inputs from the cameras and sensors, what a targetmay be and where they are located. For example, a color camera may beused by the computing system to detect a red strawberry amongst thegreen foliage of the plant it is growing on. Then a laser system couldbe used to determine a proximate location and range from the system andthe computers could use that information to triangulate a threedimensional coordinate system and identify where the target is locatedin space, relative to the traversing machine. Next, the coordinatescould be passed to the harvesting subassembly where the picker heads mayattach to and harvest the target strawberry, in some examples using itsown sensors such as cameras and lasers.

The middle-ground option between the fully automated and the manuallycontrolled system would be some variant of semi-automated seeking andharvesting. The degree of semi-autonomy and which portions wereautomated and which manually controlled could vary from separatesubassemblies. For example, the seeker subassembly may be more manuallycontrolled with a human interacting with the cameras and sensors to helpidentify targets. In some examples, that may include a human interactingwith a graphical user interface “GUI” such as a touchscreen to identifya target displayed on the screen.

In any of the above examples of automation, the sensors onboard theharvesting system may be used to create, track and pass coordinates ofthe targets for harvesting.

The computing architecture for the harvester could be described as adistributed computing system comprising of elements or processingcenters that exist on the harvester, a central server system which mayor may not be a cloud based resource and an operator processing system.Each of these processing centers are interconnected through an IPnetwork which may include local private wireless networks, private widearea networks and/or public networks such as the Internet as describedin FIGS. 13 and 14. Computational tasks may be divided such thatreal-time tasks are executed on the local harvester processor,post-processing operations and non-real time computation are executed onthe central server and user-interface computation are performed on theoperator processing center.

In example systems described herein, various computing components may beutilized to operate the systems. For example, a communication computingsystem may allow for remote operation of the machines, sensors may sendinformation to a computing system to help differentiate targets fromnon-targets, target location and mapping information may be calculated,stored, sent, and utilized between the seeker systems and harvestingsystems, steering and driving instructions may be calculated andutilized, machine learning/artificial intelligence/and/or neuralnetworks may be employed by computing systems to find and harvesttargets, and any of the other computing operations as described herein.In some examples, alternatively or additionally, a WiFi system/cellularsystem/Bluetooth system, or any other communication system, with theappropriate antenna system and a processor and memory as describedherein, may be used on a subassembly. In some embodiments, alternativelyor additionally, the hardware may include a single integrated circuitcontaining a processor core, memory, and programmable input/outputperipherals. In some examples, various computing components may be usedin the seeker and/or harvesting subassemblies, as well as thecommunication systems, control systems, and/or any other portion of thesystems described herein.

FIG. 13 shows an example networked system which could be used in thesystems and methods here. In FIG. 13, the computer system 1302 onboardthe harvesting system described herein, including process any imagesfrom the various sensors including cameras taking images of the targetsand plants. Such image data may include pixel data of the capturedtarget images. The computer 1302 could be any number of kinds ofcomputers such as those included in the camera itself, navigationsystem, robotic arm maneuvering systems, traversing or driving systems,sensor systems, and/or any other another computer arrangement includingthose examples are described in FIG. 14.

As shown in FIG. 13, the various computing systems may be incommunication with a back end computing system 1330 and/or data storage1332 to send and receive data regarding the operations of the harvestingsystems described herein. For example, as shown in FIG. 13, capturedimage data of targets may be transmitted to a back end computer system1330 and associated data storage 1332 for saving and analysis. In someexamples, this may include the remote operators who are interfacing withthe harvesting systems, selecting targets, providing navigationinstruction, steering, and/or overseeing maintenance of the systems. Insome examples, the communication may be a wireless transmission 1310 bya radio, cellular or WiFi transmission with associated routers and hubs.In some examples, the transmission may be through a wired connection1312. In some examples, a combination of wireless and wiredtransmissions may be used to stream data between the back end 1330 andthe harvesting system including cameras, robotic pickers, etc.

In some examples, the transmission of data may include transmissionthrough a network such as the internet 1320 to the remote operators,back end server computers 1330, and associated data storage 1332. Onceat the back end server computer servers 1330 and associated data storage1332, the pixelated image data may be acted upon by the remote operatorsto choose targets to harvest. In some examples, the data may be usefulto train the neural network, and/or artificial intelligence models as togood targets versus targets to pass up. In such examples, the image andtarget data may be stored, analyzed, used to train models, or any otherkind of image data analysis. In some examples, the storing, analyzing,and/or processing of image data may be accomplished at the computer 1302which is involved in the original image capture. In some examples, thelocal computer 1302 and a back end computing system 1330 may split orshare the data storing, modeling, analyzing, and/or processing. Back endcomputer resources 1330 may be more powerful, faster, or be able tohandle more data than may be otherwise available at the local computers1302 on the harvesting machines. In some examples, the networkedcomputer resources 1330 may be spread across many multiple computerresources by a cloud infrastructure. In some examples, the networkedcomputer resources 1330 may be virtual machines in a cloudinfrastructure.

FIG. 14 shows an example computing device 1400 that may be used inpracticing example embodiments described herein. FIG. 14 could describecomputers such as 1302, 1330 or other systems as described in FIG. 13.Such computing device 1400 may be the back end server systems use tointerface with the network, receive and analyzed data, as well asgenerate remote operator GUIs, additionally or alternatively, it couldbe a computing system aboard the harvesting system as described, toinstruct the robotic assemblies, cameras, communications, etc. Suchcomputer 1400 may be a device used to create and send data, as well asreceive and cause display of GUIs representing data such as back endinterfaces for remote operators, camera image analysis, etc. In FIG. 14,the computing device could be a smartphone, a laptop, tablet computer,server computer, or any other kind of computing device. The exampleshows a processor CPU 1410 which could be any number of processors incommunication via a bus 1412 or other communication with a userinterface 1414. The user interface 1414 could include any number ofdisplay devices 1418 such as a screen. The user interface also includesan input such as a touchscreen, keyboard, mouse, pointer, buttons,joystick or other input devices. Also included is a network interface1420 which may be used to interface with any wireless or wired networkin order to transmit and receive data. Such an interface may allow for asmartphone, for example, to interface a cellular network and/or WiFinetwork and thereby the Internet. The example computing device 1400 alsoshows peripherals 1424 which could include any number of otheradditional features such as but not limited to cameras, sensors 1425,and/or antennae 1426 for communicating wirelessly such as over cellular,WiFi, NFC, Bluetooth, infrared, or any combination of these or otherwireless communications. The computing device 1400 also includes amemory 1422 which includes any number of operations executable by theprocessor 1410. The memory in FIG. 14 shows an operating system 1432,network communication module 1434, instructions for other tasks 1438 andapplications 1438 such as send/receive message data 1440 and/or SMS textmessage applications 1442. Also included in the example is for datastorage 1458. Such data storage may include data tables 1460,transaction logs 1462, user data 1464 and/or encryption data 1470. Thecomputing device 1400 also include one or more graphical processingunits (GPUs) for the purposes of accelerating in hardwarecomputationally intensive tasks such as execution and or evaluation ofthe neural network engine and enhanced image exploitation algorithmsoperating on the multi-modal imagery collected. The computing device1400 may also include one or more reconfigurable hardware elements suchas a field programmable gate array (FPGA) for the purposes of hardwareacceleration of computationally intensive tasks.

CONCLUSION

As disclosed herein, features consistent with the present inventions maybe implemented by computer-hardware, software and/or firmware. Forexample, the systems and methods disclosed herein may be embodied invarious forms including, for example, a data processor, such as acomputer that also includes a database, digital electronic circuitry,firmware, software, computer networks, servers, or in combinations ofthem. Further, while some of the disclosed implementations describespecific hardware components, systems and methods consistent with theinnovations herein may be implemented with any combination of hardware,software and/or firmware. Moreover, the above-noted features and otheraspects and principles of the innovations herein may be implemented invarious environments. Such environments and related applications may bespecially constructed for performing the various routines, processesand/or operations according to the invention or they may include acomputer or computing platform selectively activated or reconfigured bycode to provide the necessary functionality. The processes disclosedherein are not inherently related to any particular computer, network,architecture, environment, or other apparatus, and may be implemented bya suitable combination of hardware, software, and/or firmware. Forexample, various machines may be used with programs written inaccordance with teachings of the invention, or it may be more convenientto construct a specialized apparatus or system to perform the requiredmethods and techniques.

Aspects of the method and system described herein, such as the logic,may be implemented as functionality programmed into any of a variety ofcircuitry, including programmable logic devices (“PLDs”), such as fieldprogrammable gate arrays (“FPGAs”), programmable array logic (“PAL”)devices, electrically programmable logic and memory devices and standardcell-based devices, as well as application specific integrated circuits.Some other possibilities for implementing aspects include: memorydevices, microcontrollers with memory (such as 1PROM), embeddedmicroprocessors, Graphics Processing Units (GPUs), firmware, software,etc. Furthermore, aspects may be embodied in microprocessors havingsoftware-based circuit emulation, discrete logic (sequential andcombinatorial), custom devices, fuzzy (neural) logic, quantum devices,and hybrids of any of the above device types. The underlying devicetechnologies may be provided in a variety of component types, e.g.,metal-oxide semiconductor field-effect transistor (“MOSFET”)technologies like complementary metal-oxide semiconductor (“CMOS”),bipolar technologies like emitter-coupled logic (“ECL”), polymertechnologies (e.g., silicon-conjugated polymer and metal-conjugatedpolymer-metal structures), mixed analog and digital, and so on.

It should also be noted that the various logic and/or functionsdisclosed herein may be enabled using any number of combinations ofhardware, firmware, and/or as data and/or instructions embodied invarious machine-readable or computer-readable media, in terms of theirbehavioral, register transfer, logic component, and/or othercharacteristics. Computer-readable media in which such formatted dataand/or instructions may be embodied include, but are not limited to,non-volatile storage media in various forms (e.g., optical, magnetic orsemiconductor storage media) and carrier waves that may be used totransfer such formatted data and/or instructions through wireless,optical, or wired signaling media or any combination thereof. Examplesof transfers of such formatted data and/or instructions by carrier wavesinclude, but are not limited to, transfers (uploads, downloads, e-mail,etc.) over the Internet and/or other computer networks by one or moredata transfer protocols (e.g., HTTP, FTP, SMTP, and so on).

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense as opposed to anexclusive or exhaustive sense; that is to say, in a sense of “including,but not limited to.” Words using the singular or plural number alsoinclude the plural or singular number respectively. Additionally, thewords “herein,” “hereunder,” “above,” “below,” and words of similarimport refer to this application as a whole and not to any particularportions of this application. When the word “or” is used in reference toa list of two or more items, that word covers all of the followinginterpretations of the word: any of the items in the list, all of theitems in the list and any combination of the items in the list.

Although certain presently preferred implementations of the inventionhave been specifically described herein, it will be apparent to thoseskilled in the art to which the invention pertains that variations andmodifications of the various implementations shown and described hereinmay be made without departing from the spirit and scope of theinvention. Accordingly, it is intended that the invention be limitedonly to the extent required by the applicable rules of law.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated. Etc.

What is claimed is:
 1. A harvesting vehicle system comprising: a vehiclewith a targeting subcomponent and a harvesting subcomponent, the vehicleincluding at least one motor in communication with wheels mounted totraverse planter bed rows; wherein the vehicle includes a computingdevice with a processor and a memory, the computing device incommunication with multiple sensors configured to generate and sendsensor data regarding agricultural targets to the computing device,wherein the computing device configured to map the agricultural targetsusing the sensor data; the harvesting subcomponent in communication withthe computing device, the harvesting subcomponent including a roboticarm with picker head assembly, the picker head assembly including avacuum hose in communication with a compressor, the hose terminating ina bellows end and spoons configured to pinch together to remove targetsusing data received from the computer regarding mapped targets.
 2. Thesystem of claim 1 further comprising, a conveyor belt system, mounted tothe harvesting subcomponent, the conveyor belt system configured toreceive and move targets from the harvesting subcomponent to a packingarea of the system.
 3. The system of claim 1 further comprising, aturbine and vacuum assembly mounted to a robotic arm of the harvestersubcomponent, the turbine and vacuum assembly in communication with thecomputer, configured to chop and suction material using data from thecomputer and map.
 4. The system of claim 1 wherein the harvestingsubcomponent includes a picker head assembly with multiple picker headsmounted to one robotic arm, the picker heads configured to rotate toharvest and deposit agricultural targets.
 5. The system of claim 1further comprising, a foliage management subcomponent mounted to theharvester subcomponent, the foliage management subcomponent includingbelt drives in communication with the computing device, the belt drivesincluding a belt configured to rotate on the belt drives, the beltconfigured to interact with foliage of the agricultural targets, andbend the foliage to reveal agricultural targets for the sensors.
 6. Thesystem of claim 5 wherein the belt drives are synchronized with the atleast one motor by the computing device to move the harvester wheels,such that a relative speed over ground of the harvester and a portion ofthe belt that is configured to contact the foliage, is close to zerowhen the harvester is in motion.
 7. The system of claim 5 wherein thefoliage management subcomponent includes a suction blower mounted abovethe foliage management subcomponent, the suction blower configured tosuck ambient air from around the target foliage, up and away fromfoliage.
 8. The system of claim 5 wherein the foliage managementsubcomponent includes a second set of belt drives and a second beltmounted around the second set of belt drives, configured to rotate onthe belt drives, wherein the two belts configured to squeeze foliage toreveal agricultural targets.
 9. The system of claim 8 wherein the secondset of belt drives are synchronized with the at least one motor by thecomputing device to move the harvester wheels, such that a relativespeed over ground of the harvester and a portion of the second belt thatis configured to contact the foliage, is close to zero when theharvester is in motion.
 10. The system of claim 5 further comprising, atleast one air blower mounted on the foliage management subcomponent, theair blower configured to blow air toward the foliage to clear debris.11. The system of claim 1 wherein the robotic arm includes a secondvacuum assembly including a hose, configured to receive the agriculturaltargets from the picker head assembly and remove the targets from theharvester subcomponent to a packing area of the system.
 12. The systemof claim 1 further comprising a back end computing system incommunication with the harvester computer, the back end computing systemconfigured to allow human users to review sensor data and designateagricultural targets for the harvester subcomponent to harvest.
 13. Thesystem of claim 1 wherein the computer uses neuro network logic to makepreliminary determinations of targets using the sensor data.
 14. Thesystem of claim 12 wherein the computer may allow the human user todetermine an angle of attack for the picker head to harvest anagricultural target, using the sensor data.
 15. A system for harvestingagriculture, comprising: a traversing machine with at least two roboticarticulating arms attached to a frame of the traversing machine, atleast two wheels or tracks attached to the frame of the traversingmachine, and a computing system with at least a processor and memoryattached to the frame of the traversing machine; the robotic armsincluding at least one picker subassembly in communication with thecomputer system; at least one sensor attached to the frame of thetraversing machine, configured to capture and send sensor informationregarding potential agricultural targets to the computing system foranalysis; the picker subassembly including a vacuum assembly coupled toa nozzle with a terminating end, wherein the terminating nozzle endincludes a segmented ported section; the picker subassembly furtherincluding two grappling spoons, the grappling spoons configured to pinchtogether toward the vacuum nozzle.
 16. The system of claim 15 whereinthe picker subassembly vacuum nozzle is mounted on an extender actuator.17. The system of claim 15 wherein the baffle section is made ofresilient, pliable material.
 18. The system of claim 15 wherein the atleast one sensor is at least one of a camera, laser, and lidar.
 19. Thesystem of claim 15 further comprising a foliage management system, thefoliage management system including two sets of belt drives, and motorsfor the belt drives in communication with the computing system, twobelts, one each around the respective belt drive set the belts eachincluding an inside and an outside, the belt roller sets and beltsconfigured on the traversing machine to receive foliage between the twobelts, on the respective insides of each, wherein the belt roller setsare synchronized with the at least two wheels or tracks such that arelative motion of an inside of the belts and ground on which thetraversing machine is moving, is approximately zero.
 20. The system ofclaim 19 wherein the foliage management system includes an ambient airvacuum pump in communication with the computing system, the vacuum pumpconfigured above the two sets of belt rollers configured to suck airaway from plant beds over which the traversing machine is configured topass.